Search for: All records

The spatial audio technique wave field synthesis has the ability to perceptually locate sound sources in front of the loudspeaker array. This can create a challenge for pairing congruent visuals when using a traditional projection screen, such as in many immersive environments. If the sound is placed within the environment beyond the boundary of the screen, how can visuals be displayed such that they appear congruent with their corresponding sound? For this, the technique known as forced perspective, where objects are made to appear of different scales or positions than they are in reality, is utilized. This allows for content to be created to appear as if it is spilling into the environment, and thus remain congruent with its respective sound. Assessments to determine perceptions and the effectiveness of audiovisual sources paired in this way are being planned and will be underway. The technique described here can be utilized for various digital experiences, from artistic performances to 3D advertisements. 

This paper proposes an experiential method for learning acoustics and consequences of room design through the rapid creation of audio-visual congruent walkable auralizations. An efficient method produces auralizations of acoustical landmarks using a two-dimensional ray-tracing algorithm and publicly available floor plans for a 128-channel wave-field synthesis system. Late reverberation parameters are calculated using additional volumetric data. Congruent visuals are produced using a web-based interface accessible via personal devices, which automatically formats for and transmits to the immersive display. Massive user-contributed online databases are harnessed through application programming interfaces, such as those offered by the Google Maps Platform, to provide near-instant access to innumerable locations. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. Listeners can walk through these recreations over an extended user area (12 m × 10 m). 

Human-scale immersive environments offer rich, often interactive, experiences and their potential has been demonstrated across areas of research, teaching, and art. The variety of these spaces and their bespoke configurations leads to a requirement for content highly-tailored to individual environments and/or interfaces requiring complicated installations. These introduce hurdles which burden users with tedious and difficult learning curves, leaving less time for project development and rapid prototyping. This project demonstrates an interactive application to control and rapid-prototype within the Collaborative-Research Augmented Immersive Virtual Environment Laboratory, or CRAIVE-Lab. Application Programming Interfaces (APIs) render complex functions of the immersive environment, such as audio spatialization, accessible via the Internet. A front-end interface configured to communicate with these APIs gives users simple and intuitive control over these functions from their personal devices (e.g. laptops, smartphones). While bespoke systems will often require bespoke solutions, this interface allows users to create content on day one, from their own devices, without set up, content-tailoring, or training. Three examples utilizing some or all of these functions are discussed. 

This paper proposes an efficient method to create auralizations of acoustical landmarks using a 2D ray-tracing algorithm and publicly available floor plans for a 128-channel wave field synthesis (WFS) system with 2.5D approximation. Late reverberation parameters are calculated using additional volumetric data. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. The listeners can walk through these recreations over an extended user area (1210 sqm), and the software suite can be used to calculate room acoustical parameters for various positions directly using a binaural rendering method or via the WFS simulation. 

This work, situated at Rensselaer's Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVELab), uses panoramic image datasets for spatial audio display. A system is developed for the room-centered immersive virtual reality facility to analyze panoramic images on a segment-by-segment basis, using pre-trained neural network models for semantic segmentation and object detection, thereby generating audio objects with respective spatial locations. These audio objects are then mapped with a series of synthetic and recorded audio datasets and populated within a spatial audio environment as virtual sound sources. The resulting audiovisual outcomes are then displayed using the facility's human-scale panoramic display, as well as the 128-channel loudspeaker array for wave field synthesis (WFS). Performance evaluation indicates effectiveness for real-time enhancements, with potentials for large-scale expansion and rapid deployment in dynamic immersive virtual environments.